DynBench: A Benchmark Suite for Dynamic Real-Time Systems

In this paper we present the architecture and framework for a benchmark suite that has been developed as part of the DeSiDeRaTa project. The benchmark suite is representative of the emerging generation of distributed, mission-critical, real-time control systems that operate in dynamic environments. Systems that operate in such environments may have unknown worstcase scenarios, may have large variances in the sizes of the data and event sets that they process (and thus, have large variances in execution latencies and resource requirements), and may be very difficult to characterize statically, even by time-invariant statistical distributions. The benchmark suite (called DynBench) is useful for evaluation of the Quality of Service (QoS) management and/or Resource Management (RM) services in distributed real-time systems. As such, DynBench includes a set of performance metrics for the evaluation of the QoS and RM technologies in dynamic, distributed real-time systems. The paper demonstrates the successful application of DynBench in evaluation of the DeSiDeRaTa QoS management middleware